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1 . REASON  FOR  ISSUANCE 

To  document  steps  taken  to  validate  Crop  Condition  Assessment 
System  hardware  and  software  from  initial  delivery  to  the  current 
time  and  to  provide  basic  guidelines  for  future  validation 
activities  and  routine  evaluation  of  output  products. 


2.  COVERAGE 

This  plan  applies  to  all  Crop  Condition  Assessment  System  hard- 
ware and  software  modifications  or  enhancements  and  all  resulting 
output  products  routinely  generated  in  support  of  documented 
mission  statements. 
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PART  1.0  BASIC  PROVISIONS 


1.1  PURPOSE 

o To  document  steps  taken  to  validate  Crop  Condition  Assessment 
System  CCCAS)  hardware  and  software  from  initial  delivery  to 
current  time. 

o To  define  activities  necessary  to  validate  system  performance  in 
terms  of  clustering  and  classification  functions. 

o To  define  general  guidelines  and  procedures  to  "be  followed  when 
additional  capabilities  are  implemented. 

o To  provide  basic  guidelines  for  routine  evaluation  of  output  pro- 
ducts generated  to  satisfy  user  requirements  within  the  confines 
of  documented  CCA  mission  statements. 


1 . 2  BACKGROUND 

Hardware  components  and  specified  software  modules  were  procured  by 
USDA.  The  CCAS  was  designated  as  a research  and  development  CR&D) 
system. 

The  prime  function  of  the  CCA  program  is  crop  condition  assessment, 
not  crop  inventory.  This  primary  function  will  be  reflected  in  output 
products  to  be  routinely  evaluated, 

A selected  group  of  USDA  analysts  received  preliminary  training  on  the 
1-100  system  located  in  Building  IT,  Johnson  Space  Center,  Initial 
analyst  training  regarding  use  of  image  analysis  software  modules  and 
computer  operator  system  training  was  conducted  by  Ford  Aerospace 
and  Communications  Corporation  (FACCl. 


1,3  PARTICIPANTS 

Ih  addition  to  USDA  personnel,  appropriate  contractor  personnel, 
specifically  representatives  of  the  Ford  Aerospace  and  Communications 
Corporation  CFACC)  and  the  Lockheed  Electronics  Corporation  (XEC)  and 
NASA  personnel  (EOD/ERPOl  will  be  invited  to  participate  in  system 
testing  and  evaluation  activities  conducted  at  the  sample  segment  level 
and  above.  Pixel  level  testing  and  evaluation  will  be  a joint  CCA/ 

FACC  activity. 
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PART  2.0  ADP  SYSTEM  VALIDATION 


2.1  OVERVIEW 

The  CCAS  has  been  subjected  to  a series  of  validation  procedures  to 
verify  ADP  consistency.  This  validation  process  will  continue  and  be 
expanded  as  new  capabilities  are  added.  The  primary  validation  testing 
is  listed  below. 

2.2  INTERNAL  CONTRACTOR  TESTING 

Prior  to  the  award  for  image  analysis  equipment,  FACC  had  tested  the 
nucleus  of  software  that  would  be  the  basis  of  image  processing  soft- 
ware for  the  CCAS.  This  testing  was  undertaken  based  on  a plan. 

Detailed  Test  Procedures  Specification  for  the  Integrated  Multivariate 

Data  Analysis  and  Classification  System  (IMDACS),  dated  March  18,  1977. 

Testing  was  concluded  in  May  and  the  image  processing  software  was 
internally  accepted  by  FACC. 

2.3  CCAS  ACCEPTANCE  TEST  PLAN  AND  RESULTS 

After  award  of  the  .image  analysis  equipment  contract  to  FACC,  USDA  and 
FACC  personnel  developed  a system  test  approach  which  relied  on  two 
methods  of  validation. 

2.3.1  System  Executive  Software.  The  system  executive  software,  such 
as  the  operating  system  and  compilers,  was  to  be  tested  by  actual 
programmer  use  prior  to  acceptance  of  the  system.  USDA  analysts/ 
programmers  began  using  the  system  for  applications  two  months 
prior  to  system  acceptance.  No  significant  deficiencies  were 
detected. 

2.3.2  Image  Processing  Software.  Image  processing  software  was  also 
evaluated  in  accordance  with  a specific  plan.  Acceptance  Test  Plan 
for  the  Single  Thread  System,  dated  July  15,  1977.  This  plan  led 
to  a detailed  procedure.  Test  Procedures  Specification,  USDA  ATS 
Integrated  Multivariate  Data  Analysis  and  Classification  System 

(iMDACS ),  dated  October  10,  1977.  IMDACS  is  a proprietary  image 

processing  package  of  FACC.  Source  code,  while  available  to  USDA, 
is  not  releasable  to  third  parties.  Therefore,  the  test  plan 
stressed  step-by-step  processing  of  images  and  output  results  so 
that  these  could  be  reviewed  by  third  parties  assisting  USDA  in 
the  evaluation. 

Testing  was  completed  on  October  kt  1977.  Thirty-seven  (37)  dis- 
crepancies were  documented  by  USDA  and  were  transmitted  to  FACC 
for  resolution.  The  one  unresolved  discrepancy  was  that  clustering/ 
classification  validation  was  a continuing  process,  that  this 
validation  must  always  - because  of  differences  in  system  archi- 
tecture - contend  with  results  differences  (CCAS  and  ERIPS), 
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However,  the  immediate  needs  of  USDA  (i.e,,  crop  analyst  .training 
and  application  programming)  were  served  by  the  CCAS  as  then  con- 
figured. Accordingly,  the  30-day  acceptance  period  began  on 
October  21,  1977. 

2.3.3  Data  Base  Management  System  (DBMS)  Software.  The  DBMS  was  accepted 
on  October  13,  1977,  using  the  IDMS  ( DBMS-11 ) Acceptance  Test 
Procedure.  No  discrepancies  were  noted  during  the  acceptance  test 
and  no  deficiencies  have  been  discovered  during  subsequent  usage. 

2.b  AGENA  BUILDING  ACCEPTANCE  TEST 

When  the  CCAS  equipment  was  moved  to  the  Agena  Building,  the  system  was 
subjected  to  the  same  acceptance  test  procedures  referenced  in  Section 
2.3.  No  new  discrepancies  were  noted. 

2.5  FUTURE  TESTING 

Future  testing  is  required  for  two  FACC  deliverables, 

2.5.1  Multistation  IMDACS  Software.  Although  testing  of  this  software 
package  is  incomplete,  one  serious  discrepancy  has  been  noted.  The 
discrepancy  is  that  one  analyst  station  can  gain  control  of  another 
station’s  images.  FACC  reported  that  the  problem  had  been  resolved, 
and  a plan  for  testing  the  resolution  of  this  discrepancy  was  avail- 
able on  July  21,  1978.  Testing  was  initiated  on  August  9,  1978. 

2.5.2  Query /Report  Writer.  A briefing  regarding  the  capabilities  of  this 
software  package  was  presented  to  CCA  programmers  and  crop  analysts 
by  FACC  on  July  31,  1978.  A test  plan  was  prepared  by  FACC,  and 
testing  was  initiated  on  August  11,  1978. 
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PART  3.0  ANALYSIS  CAPABILITIES  AND  PROGRAMS 


3 . 1 OVERVIEW 

Existing  and  planned  capabilities  of  the  CCAS  are  presented  below  in 
terms  of  computer  software  packages  available  to  support  specific 
analysts  and  administrative  functions.  Capabilities  are  stratified 
in  terms  of  those  that  are  currently  operational,  those  currently  in 
development  and  testing  phases,  and  planned  enhancements. 

3 . 2 OPERATIONAL 

Capabilities  listed  below  are  considered  oper  at  tonal , i\e.,  .initial 
testing  has  been  completed,  and  software  is  available  for  routine 
use.  Final  documentation  of  test  procedures  and  test  results  is  in 
progress. 


3.2.1  Data  Conversion  and  Load  Programs 

3. 2. 1.1  Sample  segment  batch  load  programs.  Loads  sample  segments  to 
disk  for  use  by  IMDACS, 

3. 2. 1.2  Meteorological  data  conversion.  Translates  data  received  from 
Building  IT,  JSC,  via  magnetic  tape  for  EBCDIC  to  ASCII. 

3. 2. 1.3  Tape  conversion.  Generalized  EBCDIC  to  ASCII  conversion  program. 

3. 2. 1.1  Grid  generation  program.  Product  IJ  grids  for  the  gridded  data 
base. 

3. 2. 1.5  Date  base.  Data  base  is  completely  structured  and  is  loaded 
to  country/crop  levels. 

3.2.2  Image  Analysis /Analyst  Aids 

3. 2. 2.1  Vegetation  index  computation  program.  Computes  vegetation 
indexes . 

3. 2. 2. 2 General  purpose  plot  program  locates  and  plots  the  following 
items , 


3. 2. 2. 2.1  Vegetative  index  numbers  for  a sample  segment. 

3. 2. 2. 2. 2 Soil  moisture  budget  output  from  either  gridded  data  or 
data  spread  based  on  point  observations  to  either  grids 
or  sample  segments. 
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3.2.2. 2. 3  Crop  calendar  output  from  either  gridded  data  or -data 
spread  based  on  point  observations  to  either  grids  or 
sample  segments . 

3. 2. 2. 2.1+  Sample  segment  location. 

3. 2. 2. 2. 5 Sample  segments  acquired  on  a specified  Julian  data  or  for 
fixed  periods  (e.g.,  l8-day  intervals). 

3. 2. 2. 3  Image  masking  program.  Image  masks  for  selected  acquisitions 
over  the  same  geographic  area,  on  which  vegetative ' indexes  can 
be  calculated. 


3. 2. 2.1+  Wheat  separability  system.  A series  of  computer  programs  to 
aid  in  wheat  separability  analysis.  Fifteen  software  modules 
were  obtained  from  LEC  to  produce  specified  data  plots  (e.g., 
green  number) . 


3.2.2. 5 Graphic  terminal  programs.  A series  of  programs  to  demonstrate 
graphic  terminal  capabilities  in  terms  of  plotting  functions, 
soil  information  retrieval  and  meteorological  alarm  displays. 


3.2.3  Statistical  Techniques 

3.2. 3.1  Discrimination  analysis  statistical  program.  Discriminates 
within  a population  based  on  attributes.  Has  been  used  pri- 
marily as  an  aid  in  discriminating  between  wheat  and  small 
grains. 

3. 2. 3. 2 Correlation/multiple  regression.  Primary  use  has  been  corre- 
lation  of  vegetation  indexes  and  wheat  yield, 

3. 2. 3. 3 Analysis  of  variance.  Used  primarily  for  analysis  of  vegeta- 
tion indexes  versus  area. 


3.2.1*  Administrative  Support 

3.2.1+.1  Cost  and  budget  system.  Used  extensively  to  develop  budget 

alternatives  for  USDA,  Consists  of  fourteen  0.1*1  subprograms 
under  one  executive  program, 

3.2,1+, 2 Tape  inventory  program.  Used  to  maintain  the  inventory  of 
magnetic  tapes. 


3.3  DEVELOPMENT  (TESTING) 

Software  packages  listed  below  are  in  the  developmental  stage  and  will 
be  ready  for  testing  in  the  near  future.  Appropriate  test  procedures 
■will  be  developed  and  applied  before  individual  software  packages  are 
made  available  for  routine  analytical  use. 
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3.3.1  Data  Base.  Ten  data  load  programs  and  nine  supporting  data  con- 
version programs . 

3.3.2  Yield  Models.  CCEA  I models  for  the  U.S.  and  the  U.S.S.R.  and 
Feyerherm  (KSU)  model. 

3.3.3  Meteorological  Alarm  System.  A series  of  programs  to  identify  un- 
usual meteorological  conditions  and  alert  the  analyst. 

3.3.1+  P-1  Integration.  Integration  of  P-1  into  IMDACS  will  be  a joint 

undertaking  of  USDA,  FACC  and  LEC . 

3 . b PLANNED  ENHANCEMENTS 

A number  of  anticipated  image  processing  software  enhancements  are 
summarized  in  this  section.  Order  of  listing  does  not  necessarily 
imply  implementation -priority . 


3.1+.1  IMDACS-Specific  Enhancements. 

3.1+.1.1  Replace  the  present  IMDACS  field  definition  module  with  soft- 
ware that  will  allow  up  to  29  field  vertices  (current  limit 
is  10). 

3.1+.1.2  Provide  analyst  with  the  option  of  specifying  signature 

generation  parameters  for  each  field  one  time  and  default  to 
these  parameters  for  all  fields  if  respecification  is  deemed 
unnecessary. 

3.1+.1.3  Replace  Xs  Y values  on  CRT  with  line  and  pixel  coordinates 
for  selected  pixels  and  points  (using  cursor). 

3.b.l.b  Display  a "class  summary  file"  prior  to  class  display  para- 
meter requests. 

3.1+.1.5  Default  parameters  for  iterative  and  adaptive  clustering  that 
are  compatible  with  ERIPS. 

3. b.1.6  Provide  a capability  to  display  histograms  on  the  VT-55. 

3 . b . 2 Other  Enhancements. 

3.b.2.1  Provide  for  identifying  a bounded  or  point  field  with  the  cursor. 

3,b.2.2  Provide  a capability  to  enclose  a group  of  dots  being  displayed 
in  a scatter  plot.  All  dots  enclosed  in  this  field  are  to  be 
presented  to  the  analyst  for  labeling  in  the  same  manner  as  the 
group  labeling  function  in  P-1. 
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3. U.2. 3 Generalize  the  dot  labeling  capability  to  non-grid  pixels  and 
fields . 

3.U.2.U  Allow  relabeling  of  fields  without  deleting  them. 

3. U.2. 5 Compute  the  vegetation  indexes  of  fields  for  inclusion  in 
the  scatter  plot . 

3. U.2. 6 Develop  an  improved  status  and  tracking  system  using  the  CCAS 
DBMS. 
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PART  4.0  IMAGE  PROCESSING,  TESTING  AND  VALIDATION 


4.1  OVERVIEW 


The  purpose  of  this  testing/validation  is  to  verify  that  hardware  and 
software  components  perform  specified  design  functions  correctly.  This 
is  not  an  accuracy  assessment  evaluation  of  analyst /machine  performance, 

4.2  SCOPE  OF  TESTING 


Clustering  and  classification  are  the  primary  elements  to  "be  tested 
against  documented  design  specifications.  Validation  of  segment-level 
results  will  he  a joint  effort  of  USDA,  NASA,  FACC  and  LEC . Pixel- 
level  tests  will  he  performed  internally  hy  USDA  and  FACC.  Continuing 
validation  of  IMDACS  results  with  results  obtained  from  ERIPS  is  anti- 
cipated, 

4 . 3__  RESPONSIBILITIES 

4.3.1  USDA  Crop  Condition  Assessment  System  (CCAS) 

4. 3. 1.1  Select  candidate  segments  for  test. 

4. 3. 1.2  In  conjunction  with  FACC,  define  the  ERIPS/IMDACS  clustering 
and  classification  parameters  to  he  used. 

4. 3. 1.3  Define  the  test  segment  runs  to  he  made  on  ERIPS. 

4. 3. 1.4  Deliver  and  pick  up  data  from  Buildings  IT  and  30,  JSC. 

4. 3. 1.5  Perform  the  lead  role  in  analysis  of  results;  required  support 
to  he  provided  hy  FACC. 

4 .3.1.6  Establish  schedule  dates  for  test  completion  and  for  correction 
action  hy  FACC  as  required. 

4. 3.1.7  Overall  management  of  test  and  documentation  of  task  procedures 
and  results  to  USDA  accepted  standard. 

4. 3. 1.8  Provide  status  reports  as  agreed  to  hy  USDA  and  NASA  management. 

4.3.2  NASA,  EARTH  OBSERVATIONS  DIVISION,  JSC 

4. 3. 2.1  Provide  current  documentation  for  LACIE  operational  iterative 
clustering  options  used  on  all  data  sets  in  the  1976-77  crop 
year. 

4. 3. 2. 2 Provide  one  test  image  used  in  ERIPS  validation. 

4. 3. 2. 3 Provide  Landsat  data  for  selected  segments;  images  should  he 
unloaded  to  tape  and  all  acquisitions  for  each  segment  merged. 

It  is  desirable  to  have  all  images  unloaded  and  merged  on  one 
tape . 
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4. 3. 2. 4 Perform  no  more  than  34  computer  runs  on  selected  inhouse  data. 
Diagnostic  listings  from  clustering  and  classification  are 
desirable  if  no  resource  impact  is  incurred  in  their  production. 

4. 3. 2. 5 Notify  the  CCAS  task  leader  when  data  are  ready. 

4. 3. 2. 6 Provide  two  man  days  of  consultation  on  documentation  and  clari- 
fication of  ERIPS  results. 


4.3.3  Ford  Aerospace  and  Communications  Corporation 

4. 3. 3.1  Prepare  detailed  test  plan,  obtain  USDA  approval  of  test  plan, 
and  perform  all  IMDACS  test  processing  with  the  USDA  task  leader 
as  a full-time  observer  and  test  director. 

4. 3. 3. 2 Thoroughly  document  analysis  performed  leading  to  test  results 
delivered  to  USDA. 

4. 3. 3. 3 Assist  in  defining  ERIPS  runs  (especially  option  selections). 

4. 3.3.4  Assist  USDA  in  performing  comparability  analysis  and  in  docu- 
menting results. 

4.4  MEASURES  OF  PERFORMANCE 

4.4.1  Segment-level  results  for  both  clustering  and  classification  must  be 
comparable  to  the  significant  digit  (whole  percent).  Test  success 
or  failure  will  not  be  judged  on  intermediate  results  (i.e.,  results 
below  the  segment  level). 

4.4.2  Availability/non-availability  of  identical  clustering  options  in 
ERIPS  and  IMDACS  will  not  be  used  as  a measure  of  success/failure 
in  this  test. 

4.4.3  Repeatability  of  procedures  to  achieve  comparable  results  is  critical. 

4 . 5 SCHEDULES 

All  schedules  will  be  predicated  on  established  USDA/FACC  priorities  and 
negotiations  under  the  existing  contract  structure. 
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PART  5.0  ACCURACY  ASSESSMENT 


5.1  LEVEL  OF  TESTING 

5.1.1  First  Order  Tests.  To  evaluate  methods,  procedures  and  algorithms 
implemented  from  LACIE  and/or  developed  by  CCA  personnel  against 
predefined  end-user  information  requirements  in  terms  of  accuracy, 
timeliness  and  continuity  of  data, 

5.1.2  Second  Order  Test . To  determine  those  system  components  Methods, 
procedures,  algorithms]  that  do  not  meet,  or  that  contribute  to  the 
inability  of  the  CCAS  to  meet  predefined  information  requirements. 

5.1.3  Third  Order  Test.  To  compare  CCAS  results,  from  methods,  procedures 
and  algorithms  transferred  from  the  LACIE  for  consistency  with  LACIE 
results  using  the  same  procedures  and  to  document  that  LACIE  trans- 
ferred technology  yields  LACIE  expected  results. 

5.2  ELEMENTS  TO  BE  TESTED 

5.2.1  Comparison  of  LACIE  and  CCAS  Segment  Level  Estimation.  The  objective 
of  this  test  is  to  compare  the  small  grains  proportion  estimate  pro- 
duced by  the  CCAS  with  that  produced  by  LACIE.  All  blind  sites  in 
North  Dakota  and  Montana  for  the  1978  crop  year  will  be  used  for 
this  test.  The  proportional  estimates  from  both  systems  on  identical 
data  sets  will  be  compared  to  actual  ground  truth  segments.  Evalua- 
tion of  the  absolute  error  in  the  small  grains  proportion  estimate 
for  both  the  CCAS  and  LACIE  will  be  reported.  If  any  significant 
differences  between  the  two  systems  exist,  when  the  segment  level 
proportion  estimates  are  made,  these  differences  will  be  reported. 

5.2.2  Comparison  of  CCA  procedures.  The  objective  of  this  test  is  to  com- 
pare the  accuracy  of  the  analyst  selected  fields  procedure  with  the 
accuracy  of  the  P-1  procedure. 

Thirty  Phase  III  blind  sites  from  North  Dakota  and  Montana  will  be 
selected.  They  should  exhibit  the  range  of  characteristics  expected 
in  the  spring  wheat  area  of  the  northern  Great  Plains.  These  sites 
will  be  grouped  into  five  data  sets  of  six  types  reflecting  levels 
of  various  factors  hypothesized  to  affect  accuracy.  Examples  of 
such  factors  are  the  actual  wheat  or  small  grains  proportion,  acreage, 
field  size,  and  the  presence  of  confusion  crops.  The  five  segments 
for  each  type  should  be  similar  enough  so  that  there  is  not  an  a 
priori  reason  to  expect  one  to  be  easier  or  more  difficult  to  analyze 
than  others  of  that  type. 

For  each  segment  processed,  the  analysts  shall  in  a single  session 
choose  and  label  all  analyst  selected  fields  and  dots  for  P-1.  The 
order  of  processing  will  be  specified  prior  to  testing.  Once  the 
labels  are  selected,  procedures  will  be  applied  to  the  segment  to 
give  two  small  grains  proportion  estimates. 
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The  only  variable  to  be  checked  is  the  absolute  error  in  .the  small 
grains  proportion  estimate  as  compared  to  available  ground  truth. 
Inferential  tests  to  identify  significant  differences  between  pro- 
cedures will  be  made;  specifically,  analysis  of  variance  techniques 
will  be  applied  to  data  obtained  from  the  experiment. 

5.2.3  Analysts  Labeling  Accuracy.  The  objective  of  this  test  is  to  eval- 
uate the  learning  curve  for  experienced  and  inexperienced  CCA  analysts 
as  the  crop  year  progresses  in  terms  of  labeling  accuracies  and 
variabilities.  Each  analyst  will  be  required  to  process  three  blind 
sites  from  Phase  III  for  evaluation  prior  to  processing  any  data 
for  the  1978  crop  year.  After  the  initial  data  set  is  evaluated,  each 
analyst  will  process  additional  data  sets  through  the  growing  season 
as  part  of  normal  operations  for  evaluation.  Appropriate  statistical 
techniques  will  be  used  to  test  for  significant  differences  between 
analysts . 

5.2.U  Yield  Models.  Two  yield  models  (.CCEA  I and  Feyerherm-KSU J developed 
and  run  in  LACIE  are  to  be  implemented  and  tested. 


5.2.U.1  CCEA  I Models.  The  CCEA  I regression  models  using  monthly  average 
temperatures  and  monthly  total  precipitation  will  be  tested. 

Phase  III  operational  model  results  will  be  compared  with  results 
obtained  from  Phase  III  models  implemented  by  the  CCAS  unit  using 
identical  input  values.  Transition  Year  operational  model  results 
(models  implemented  April  1,  1978,  by  LACIE  for  the  U.S.  Great 
Plains ) will  be  compared  monthly  with  results  of  the  CCA- implemented, 
updated  version  of  the  CCEA  I models  using  current  year  inputs. 

5. 2.4.2  Feyerherm  IKSU ) Model . The  Feyerherm  model,  as  implemented  in 
the  CCAS,  will  be  validated  by  comparing  Phase  III  LACIE  output 
for  North  Dakota  and  Tselinograd  to  CCA- implemented  model  re- 
sults using  identical  inputs  to  be  furnished  by  EOD.  The  winter 
wheat  model  will  be  implemented  and  tested  upon  receipt  of  new 
version  and  test  inputs  from  EOD. 

5.2.1+.3  Output  Products.  CCEA  I yield  models  will  be  operated  over  the 
U.S.S.R.  for  both  winter  and  spring  wheat  during  the  1977-78 
crop  season.  Yield  estimates  will  be  generated  monthly  begin- 
ning in  mid-April  (weather  data  through  March)  for  winter  wheat 
and  mid-May  (weather  data  through  April)  for  spring  wheat. 

Information  provided  for  each  crop  region  will  include  the 
following  items : 


o 

o 


Current  month  yield  prediction 
Past  monthCs)  yield  prediction 
Yield  change  from  previous  month 
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o Specific  weather  phenomenona  causing  change 

o Past  month(s)  weather  (temeprature  and  precipitation  deviation 
from  normal) 

o Cumulative  seasonal  impact  of  weather  with  breakdown  ^ tempera- 
ture and  precipitation)  on  yield  by  month 

o Comparison  of  current  month  yield  with  one  year  ago 

o Comparison  of  current  month  yield  with  trend  (percent  deviation) 

5.2.5  Crop  Calendar.  The  wheat  crop  calendar,  as  developed  by  Robertson  and 
implemented  by  Feyerherm  as  an  integral  part  of  his  yield  model,  will 
be  tested.  Daily  input  data  derived  by  LEC  as  input  into  the  KSU 
yield  model  tests  for  the  1975  and  1976  crop  years  in  Kansas  and 
North  Dakota  will  be  run  through  the  CCA-implemented  crop  calendar. 
CCA-generated  daily  growth  stage  results  will  be  compared  with  LACIE- 
generated  results.  This  test  will  include  validation  of  the  spring 
wheat  starter  model. 

5.2.6  Crop  Condition  Assessment.  The  objective  is  to  test  and  evaluate 
the  effectiveness  of  selected  procedures  in  detecting  the  existence 
of  conditions  that  may  affect  crop  area,  yield  and/or  production  and 
in  quantifying  the  impact  of  the  identified  conditions  in  terms  of 
direction  and  estimated  magnitude. 

5. 2.6.1  Climatic  Alarm  System.  This  system,  as  currently  structured, 
utilizes  automated  meteorological  parameters  to  flag  adverse 
climatic  conditions  that  affect  the  growth,  development  and 
ultimate  condition  of  the  crop  throughout  the  growing  season. 

This  capability  provides  a means  of  monitoring  current  year  condi- 
tions  and  assessing  the  year-to-year  variability  in  meteorological 
parameters  that  may  have  an  adverse  effect  on  crop  area,  yield 
and/or  production. 

A series  of  reports  will  be  generated  for  selected  areas  of  the 
U.S.  and  the  U.S.S.R.  to  alert  analysts /users  regarding  the 
existence  and  extent  of  specified  phenomena.  The  first  report 
will  indicate  that  a significant  departure  from  normal  climatic 
conditions  has  occurred  or  that  a specified  combination  of 
conditions  has  been  detected.  A second  report,  derived  from 
monitoring  an  event  over  time,  will  confirm  the  existence  of  a 
problem  and  will  provide  the  areal  extent  via  an  isoline  map 
based  on  satellite  data.  A third  report  will  qualitatively 
assess  the  impact  of  the  event  and  will  portray  supportive  data 
in  the  form  of  isoline  maps  for  the  following  elements;  Cl 1 
soil  moisture,  (2)  precipitation,  (3)  temperature,  and  Q+l 
overlay  of  vegetation  index(es). 
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5. 2. 6. 2 Vegetation  Indexes.  The  objective  is  to  test  and  evaluate 

selected  methods  for  computing  vegetative  index  numbers  IVIN). 

An  intermediate  step  vill  be  to  correlate  vegetative  index 
numbers  at  the  sample  segment  level  with  crop  growth  stages , 
soil  moisture,  crop  yield  and  crop  biomass. 

Stress  line  geographic  plots  will  be  produced  to  reflect  the 
current  vegetative  index  for  a given  area.  These  plots  will 
be  overlaid  on  a map  of  the  related  area.  The  VIN  and  associated 
plots  will  provide  the  user  an  indication  of  vegetative  condi- 
tion and  a correlation  with  soil  moisture  which  can  be  related 
to  growth  stages  of  crops  in  the  area  being  analyzed.  VIN  plots 
will  be  generated  at  spring  greenup  and  will  continue  through 
harvest  for  U.S.S.R.  spring  wheat  areas  and  for  selected  areas 
in  the  U.S. 

5.2.7  Data  Base.  To  test  and  evaluate  the  accuracy  of  the  climate,  soils, 
ag/non-ag,  and  historical  data  to  be  loaded  at  the  25x25  nm  grid 
cell  level,  and  to  each  quadrant  where  possible. 

5.2.8  Agrophysical  Unit  (APU)  Maps.  These  maps  consist  of  delineations 
of  land  areas  of  relatively  homogeneous  agricultural  use,  density 
and  potential  in  relation  to  soils,  climate,  topography,  and  agro- 
nomic practices. 

The  unit  is  commonly  referred  to  as  an  APU.  This  type  of  regionali- 
zation is  highly  supportive  of  crop  condition  analysis  as  related 
to  the  total  range  of  existing  circumstances  that  impact  crop  pro- 
duction. This  year’s  maps  will  cover  selected  areas  of  the  U.S.S.R. 
and  U.S. 

5 . 3 DATA  REQUIREMENTS 

The  evaluation  functions  will  require  supporting  data  for  North  Dakota, 

Montana  and  one  APU  in  the  U.S.S.R.  from  the  following  categories: 

o ITS 

o Blind  Sites 

o Historical  estimates 

o Current  estimates  and  periodical  reports 

o Historical  and  current  MSS  data 

o Historic  and  current  meteorological  summaries  and  climate  crop 
assessment 

o Segment , APU,  political  overlays 

o Soils  maps  and  related  classification  ag/non-ag  overlays 
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In  addition  to  the  data  specified  above,  data  from  other  areas  will  be 
required  to  support  training  and  procedures  development  tasks. 

5 . h EVALUATION  REPORTS 

Each  report  will  be  a stand-alone  document  following  the  outline  listed 
below : 

o Evaluation  Report  Number 
o Task  description 
o Task  requirements 
o Task  summary  and  objectives 
o Statistical  comparisons 
o Statistical  test 
o Task  evaluation  results 
o Review  and  recommendations 
5 . 5 MANAGEMENT 

5.5.1  Task  Manager.  This  responsibility  will  be  within  the  CCA  unit.  It 
is  the  task  manager’s  responsibility  to  assign  task  monitors,  re- 
commend task  leaders  and  candidate  task  members.  Along  with  Task 
Review  Team,  accept  and/or  recommend  additional  action  needed  for 
each  task  upon  completion  of  review. 

5.5- 2  Task  Monitor.  This  responsibility  will  be  within  the  CCA  unit.  It 

is  the  responsibility  of  the  Task  Monitor  to  maintain  and  report  on 
resources  and  schedule  changes  and  to  work  interfaces  within  the 
CCA  unit  and  other  LACIE  organizations,  assist  task  leader  as 
necessary  in  performance  of  evaluations. 

5.5*3  Task  Leader.  This  responsibility  will  be  assigned  to  a professional 
from  any  agency  participating  in  the  experiment.  It  is  the  Task 
Leader’s  responsibility  to  manage  the  technical  activity  of  the 
Task  Team  in  an  efficient  and  orderly  evaluation. 

5.5- ^  Task  Team.  Each  team  member  will  be  selected  based  upon  needed 

skills  and  project  experience  required  to  properly  complete  each 
task  evaluation,  from  various  participating  agencies. 
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5 . 6 INTERFACES 

Most  interfaces  will  occur  in  the  normal  day-to-day  data  flow  between 
NASA,  NOAA  and  USDA.  Additional  interface  requirements  will  be  developed 
for  specific  tasks  as  the  need  arises.  Initially,  the  major  impact  will 
be  within  the  CCA  unit  in  terms  of  task  development  and  evaluation. 

5-6.1  External  Interfaces.  A close  working  relationship  with  LACIE 

Accuracy  Assessment  personnel  (.civil  servants  and  support  contractors) 
is  necessary  to  optimize  use  of  available  resources.  Interfaces 
with  other  EOD  and  ERPO  personnel  will  be  on  an  as  needed  basis. 

5.6.2  Internal  Interfaces.  Regular  interaction  with  CCA  personnel  in  the 
ADP  support  group  and  the  analyst  unit  will  be  necessary  to  perform 
routine  product  evaluations  and  system  performance  tests. 
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1.  Detailed  Test  Procedures  Specification  for  the  Integrated  Multivariate 

Data  Analysis  and  Classification  System  (IMDACS),  dated  March  18,  1977. 


2.  Acceptance  Test  Plan  for  the  Single  Thread  System,  dated  July  15,  1977- 

3.  Test  Procedures  Specification,  USDA  ATS  Integrated  Multivariate  Data 

Analysis  and  Classification  System  IIMDACS),  dated  October  10,  1977. 

it.  IDMS  ( DBMS-11 ) Acceptance  Test  Procedure,  dated  October  13,  1977. 

5.  Memorandum  LACIE-78-538 , dated  April  21,  1978,  subject  "ATS  Data 

Requirements  for  1978." 

6.  Memorandum  LACIE-78-196,  dated  July  17,  1978,  subject  "Test  and  Evalua- 

tion of  the  USDA-IMDACS  Clustering  and  Classification  System." 

7.  Memorandum  LACIE-78-625 , dated  July  l8,  1978,  subject  "ERIPS/IMDACS 

Test-Image  Unload  and  Merge." 

8.  NASA  Memorandum  SF-12 , dated  July  13,  1978,  subject  "USDA  Requested 

Actions  Related  to  ATS  Systems  Verification." 

9.  NASA  Memorandum  SF-12,  dated  July  19,  1978,  subject  "ERIPS  Iterative 

Clustering  Documentation." 
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